{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7765UFHoyGx6"
      },
      "source": [
        "##### Copyright 2020 The TensorFlow Authors."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "KsOkK8O69PyT"
      },
      "outputs": [],
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZS8z-_KeywY9"
      },
      "source": [
        "# TFL レイヤーを使用した Keras モデルの作成"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r61fkA2i9Y3_"
      },
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td><a target=\"_blank\" href=\"https://www.tensorflow.org/lattice/tutorials/keras_layers\"><img src=\"https://www.tensorflow.org/images/tf_logo_32px.png\"> TensorFlow.orgで表示</a></td>\n",
        "  <td><a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs-l10n/blob/master/site/ja/lattice/tutorials/keras_layers.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\"> Google Colab で実行</a></td>\n",
        "  <td><a target=\"_blank\" href=\"https://github.com/tensorflow/docs-l10n/blob/master/site/ja/lattice/tutorials/keras_layers.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\">GitHub でソースを表示{</a></td>\n",
        "  <td><a href=\"https://storage.googleapis.com/tensorflow_docs/docs-l10n/site/ja/lattice/tutorials/keras_layers.ipynb\"><img src=\"https://www.tensorflow.org/images/download_logo_32px.png\">ノートブックをダウンロード/a0}</a></td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ecLbJCvJSSCd"
      },
      "source": [
        "##概要\n",
        "\n",
        "TFL Keras レイヤーを使用すると、単調性制約や他の形状制約を持つ Keras モデルを構築できます。この例では、TFL レイヤーを使用して、UCI 心臓データセットのキャリブレーションされた格子モデルを構築してトレーニングします。\n",
        "\n",
        "キャリブレーションされた格子モデルでは、各特徴は`tfl.layers.PWLCalibration`または`tfl.layers.CategoricalCalibration`レイヤーによって変換され、結果は`tfl.layers.Lattice`を使用して非線形的に融合されます。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x769lI12IZXB"
      },
      "source": [
        "## セットアップ"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fbBVAR6UeRN5"
      },
      "source": [
        "TF Lattice パッケージをインストールします。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bpXjJKpSd3j4"
      },
      "outputs": [],
      "source": [
        "#@test {\"skip\": true}\n",
        "!pip install tensorflow-lattice pydot"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jSVl9SHTeSGX"
      },
      "source": [
        "必要なパッケージをインポートします。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "both",
        "id": "pm0LD8iyIZXF"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "\n",
        "import logging\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import sys\n",
        "import tensorflow_lattice as tfl\n",
        "from tensorflow import feature_column as fc\n",
        "logging.disable(sys.maxsize)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "svPuM6QNxlrH"
      },
      "source": [
        "UCI Statlog (心臓) データセットをダウンロードします。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "both",
        "id": "PG3pFtK-IZXM"
      },
      "outputs": [],
      "source": [
        "# UCI Statlog (Heart) dataset.\n",
        "csv_file = tf.keras.utils.get_file(\n",
        "    'heart.csv', 'http://storage.googleapis.com/applied-dl/heart.csv')\n",
        "training_data_df = pd.read_csv(csv_file).sample(\n",
        "    frac=1.0, random_state=41).reset_index(drop=True)\n",
        "training_data_df.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nKkAw12SxvGG"
      },
      "source": [
        "このガイドのトレーニングに使用するデフォルト値を設定します。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "both",
        "id": "krAJBE-yIZXR"
      },
      "outputs": [],
      "source": [
        "LEARNING_RATE = 0.1\n",
        "BATCH_SIZE = 128\n",
        "NUM_EPOCHS = 100"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0TGfzhPHzpix"
      },
      "source": [
        "## Sequential Keras モデル\n",
        "\n",
        "この例では、Sequential Keras モデルを作成し、TFL レイヤーのみを使用します。\n",
        "\n",
        "Lattice レイヤーは、`input [i]`が`[0、lattic_sizes [i]-1.0]`内にあることを期待しているため、キャリブレーションレイヤーの前にラティスサイズを定義して、キャリブレーションレイヤーの出力範囲を適切に指定します。\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nOQWqPAbQS3o"
      },
      "outputs": [],
      "source": [
        "# Lattice layer expects input[i] to be within [0, lattice_sizes[i] - 1.0], so\n",
        "lattice_sizes = [3, 2, 2, 2, 2, 2, 2]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W3DnEKWvQYXm"
      },
      "source": [
        "<code>tfl.layers.ParallelCombination</code>レイヤーを使用して、Sequential モデルを作成できるようにするために並行して実行する必要があるキャリブレーションレイヤーをグループ化します。\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "o_hyk5GkQfl8"
      },
      "outputs": [],
      "source": [
        "combined_calibrators = tfl.layers.ParallelCombination()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BPZsSUZiQiwc"
      },
      "source": [
        "各特徴のキャリブレーションレイヤーを作成し、それを並行コンビネーションレイヤーに追加します。数値特徴の場合は`tfl.layers.PWLCalibration`、カテゴリカル特徴の場合は`tfl.layers.CategoricalCalibration`を使用します。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DXPc6rSGxzFZ"
      },
      "outputs": [],
      "source": [
        "# ############### age ###############\n",
        "calibrator = tfl.layers.PWLCalibration(\n",
        "    # Every PWLCalibration layer must have keypoints of piecewise linear\n",
        "    # function specified. Easiest way to specify them is to uniformly cover\n",
        "    # entire input range by using numpy.linspace().\n",
        "    input_keypoints=np.linspace(\n",
        "        training_data_df['age'].min(), training_data_df['age'].max(), num=5),\n",
        "    # You need to ensure that input keypoints have same dtype as layer input.\n",
        "    # You can do it by setting dtype here or by providing keypoints in such\n",
        "    # format which will be converted to deisred tf.dtype by default.\n",
        "    dtype=tf.float32,\n",
        "    # Output range must correspond to expected lattice input range.\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[0] - 1.0,\n",
        ")\n",
        "combined_calibrators.append(calibrator)\n",
        "\n",
        "# ############### sex ###############\n",
        "# For boolean features simply specify CategoricalCalibration layer with 2\n",
        "# buckets.\n",
        "calibrator = tfl.layers.CategoricalCalibration(\n",
        "    num_buckets=2,\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[1] - 1.0,\n",
        "    # Initializes all outputs to (output_min + output_max) / 2.0.\n",
        "    kernel_initializer='constant')\n",
        "combined_calibrators.append(calibrator)\n",
        "\n",
        "# ############### cp ###############\n",
        "calibrator = tfl.layers.PWLCalibration(\n",
        "    # Here instead of specifying dtype of layer we convert keypoints into\n",
        "    # np.float32.\n",
        "    input_keypoints=np.linspace(1, 4, num=4, dtype=np.float32),\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[2] - 1.0,\n",
        "    monotonicity='increasing',\n",
        "    # You can specify TFL regularizers as a tuple ('regularizer name', l1, l2).\n",
        "    kernel_regularizer=('hessian', 0.0, 1e-4))\n",
        "combined_calibrators.append(calibrator)\n",
        "\n",
        "# ############### trestbps ###############\n",
        "calibrator = tfl.layers.PWLCalibration(\n",
        "    # Alternatively, you might want to use quantiles as keypoints instead of\n",
        "    # uniform keypoints\n",
        "    input_keypoints=np.quantile(training_data_df['trestbps'],\n",
        "                                np.linspace(0.0, 1.0, num=5)),\n",
        "    dtype=tf.float32,\n",
        "    # Together with quantile keypoints you might want to initialize piecewise\n",
        "    # linear function to have 'equal_slopes' in order for output of layer\n",
        "    # after initialization to preserve original distribution.\n",
        "    kernel_initializer='equal_slopes',\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[3] - 1.0,\n",
        "    # You might consider clamping extreme inputs of the calibrator to output\n",
        "    # bounds.\n",
        "    clamp_min=True,\n",
        "    clamp_max=True,\n",
        "    monotonicity='increasing')\n",
        "combined_calibrators.append(calibrator)\n",
        "\n",
        "# ############### chol ###############\n",
        "calibrator = tfl.layers.PWLCalibration(\n",
        "    # Explicit input keypoint initialization.\n",
        "    input_keypoints=[126.0, 210.0, 247.0, 286.0, 564.0],\n",
        "    dtype=tf.float32,\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[4] - 1.0,\n",
        "    # Monotonicity of calibrator can be decreasing. Note that corresponding\n",
        "    # lattice dimension must have INCREASING monotonicity regardless of\n",
        "    # monotonicity direction of calibrator.\n",
        "    monotonicity='decreasing',\n",
        "    # Convexity together with decreasing monotonicity result in diminishing\n",
        "    # return constraint.\n",
        "    convexity='convex',\n",
        "    # You can specify list of regularizers. You are not limited to TFL\n",
        "    # regularizrs. Feel free to use any :)\n",
        "    kernel_regularizer=[('laplacian', 0.0, 1e-4),\n",
        "                        tf.keras.regularizers.l1_l2(l1=0.001)])\n",
        "combined_calibrators.append(calibrator)\n",
        "\n",
        "# ############### fbs ###############\n",
        "calibrator = tfl.layers.CategoricalCalibration(\n",
        "    num_buckets=2,\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[5] - 1.0,\n",
        "    # For categorical calibration layer monotonicity is specified for pairs\n",
        "    # of indices of categories. Output for first category in pair will be\n",
        "    # smaller than output for second category.\n",
        "    #\n",
        "    # Don't forget to set monotonicity of corresponding dimension of Lattice\n",
        "    # layer to '1'.\n",
        "    monotonicities=[(0, 1)],\n",
        "    # This initializer is identical to default one('uniform'), but has fixed\n",
        "    # seed in order to simplify experimentation.\n",
        "    kernel_initializer=tf.keras.initializers.RandomUniform(\n",
        "        minval=0.0, maxval=lattice_sizes[5] - 1.0, seed=1))\n",
        "combined_calibrators.append(calibrator)\n",
        "\n",
        "# ############### restecg ###############\n",
        "calibrator = tfl.layers.CategoricalCalibration(\n",
        "    num_buckets=3,\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[6] - 1.0,\n",
        "    # Categorical monotonicity can be partial order.\n",
        "    monotonicities=[(0, 1), (0, 2)],\n",
        "    # Categorical calibration layer supports standard Keras regularizers.\n",
        "    kernel_regularizer=tf.keras.regularizers.l1_l2(l1=0.001),\n",
        "    kernel_initializer='constant')\n",
        "combined_calibrators.append(calibrator)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "inyNlSBeQyp7"
      },
      "source": [
        "次に、キャリブレータの出力を非線形に融合するラティスレイヤーを作成します。\n",
        "\n",
        "必要とされる次元で増加するラティスの単調性を指定する必要があることに注意してください。キャリブレーションで単調性の方向を持つ構成は、単調性の正しいエンドツーエンドの方向になります。これには、CategoricalCalibration レイヤーの部分的な単調性が含まれます。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DNCc9oBTRo6w"
      },
      "outputs": [],
      "source": [
        "lattice = tfl.layers.Lattice(\n",
        "    lattice_sizes=lattice_sizes,\n",
        "    monotonicities=[\n",
        "        'increasing', 'none', 'increasing', 'increasing', 'increasing',\n",
        "        'increasing', 'increasing'\n",
        "    ],\n",
        "    output_min=0.0,\n",
        "    output_max=1.0)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T5q2InayRpDr"
      },
      "source": [
        "次に、キャリブレータとラティスレイヤーを組み合わせて Sequential モデルを作成します。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xX6lroYZQy3L"
      },
      "outputs": [],
      "source": [
        "model = tf.keras.models.Sequential()\n",
        "model.add(combined_calibrators)\n",
        "model.add(lattice)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W3UFxD3fRzIC"
      },
      "source": [
        "トレーニングは、他の Keras モデルと同じように機能します。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2jz4JvI-RzSj"
      },
      "outputs": [],
      "source": [
        "features = training_data_df[[\n",
        "    'age', 'sex', 'cp', 'trestbps', 'chol', 'fbs', 'restecg'\n",
        "]].values.astype(np.float32)\n",
        "target = training_data_df[['target']].values.astype(np.float32)\n",
        "\n",
        "model.compile(\n",
        "    loss=tf.keras.losses.mean_squared_error,\n",
        "    optimizer=tf.keras.optimizers.Adagrad(learning_rate=LEARNING_RATE))\n",
        "model.fit(\n",
        "    features,\n",
        "    target,\n",
        "    batch_size=BATCH_SIZE,\n",
        "    epochs=NUM_EPOCHS,\n",
        "    validation_split=0.2,\n",
        "    shuffle=False,\n",
        "    verbose=0)\n",
        "\n",
        "model.evaluate(features, target)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RTHoW_5lxwT5"
      },
      "source": [
        "## Functional Keras モデル\n",
        "\n",
        "この例では、Keras モデルの構築に Functional  API を使用しています。\n",
        "\n",
        "前述のとおり、ラティスレイヤーは、`input [i]`が`[0、lattic_sizes [i]-1.0]`内にあることを期待しているため、キャリブレーションレイヤーの前にラティスサイズを定義して、キャリブレーションレイヤーの出力範囲を適切に指定します。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gJjUYvBuW1qE"
      },
      "outputs": [],
      "source": [
        "# We are going to have 2-d embedding as one of lattice inputs.\n",
        "lattice_sizes = [3, 2, 2, 3, 3, 2, 2]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z03qY5MYW1yT"
      },
      "source": [
        "各特徴ごとに入力レイヤーを作成してからキャリブレーションレイヤーを作成する必要があります。数値特徴の場合は`tfl.layers.PWLCalibration`、カテゴリカル特徴の場合は`tfl.layers.CategoricalCalibration`を使用します。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DCIUz8apzs0l"
      },
      "outputs": [],
      "source": [
        "model_inputs = []\n",
        "lattice_inputs = []\n",
        "# ############### age ###############\n",
        "age_input = tf.keras.layers.Input(shape=[1], name='age')\n",
        "model_inputs.append(age_input)\n",
        "age_calibrator = tfl.layers.PWLCalibration(\n",
        "    # Every PWLCalibration layer must have keypoints of piecewise linear\n",
        "    # function specified. Easiest way to specify them is to uniformly cover\n",
        "    # entire input range by using numpy.linspace().\n",
        "    input_keypoints=np.linspace(\n",
        "        training_data_df['age'].min(), training_data_df['age'].max(), num=5),\n",
        "    # You need to ensure that input keypoints have same dtype as layer input.\n",
        "    # You can do it by setting dtype here or by providing keypoints in such\n",
        "    # format which will be converted to deisred tf.dtype by default.\n",
        "    dtype=tf.float32,\n",
        "    # Output range must correspond to expected lattice input range.\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[0] - 1.0,\n",
        "    monotonicity='increasing',\n",
        "    name='age_calib',\n",
        ")(\n",
        "    age_input)\n",
        "lattice_inputs.append(age_calibrator)\n",
        "\n",
        "# ############### sex ###############\n",
        "# For boolean features simply specify CategoricalCalibration layer with 2\n",
        "# buckets.\n",
        "sex_input = tf.keras.layers.Input(shape=[1], name='sex')\n",
        "model_inputs.append(sex_input)\n",
        "sex_calibrator = tfl.layers.CategoricalCalibration(\n",
        "    num_buckets=2,\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[1] - 1.0,\n",
        "    # Initializes all outputs to (output_min + output_max) / 2.0.\n",
        "    kernel_initializer='constant',\n",
        "    name='sex_calib',\n",
        ")(\n",
        "    sex_input)\n",
        "lattice_inputs.append(sex_calibrator)\n",
        "\n",
        "# ############### cp ###############\n",
        "cp_input = tf.keras.layers.Input(shape=[1], name='cp')\n",
        "model_inputs.append(cp_input)\n",
        "cp_calibrator = tfl.layers.PWLCalibration(\n",
        "    # Here instead of specifying dtype of layer we convert keypoints into\n",
        "    # np.float32.\n",
        "    input_keypoints=np.linspace(1, 4, num=4, dtype=np.float32),\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[2] - 1.0,\n",
        "    monotonicity='increasing',\n",
        "    # You can specify TFL regularizers as tuple ('regularizer name', l1, l2).\n",
        "    kernel_regularizer=('hessian', 0.0, 1e-4),\n",
        "    name='cp_calib',\n",
        ")(\n",
        "    cp_input)\n",
        "lattice_inputs.append(cp_calibrator)\n",
        "\n",
        "# ############### trestbps ###############\n",
        "trestbps_input = tf.keras.layers.Input(shape=[1], name='trestbps')\n",
        "model_inputs.append(trestbps_input)\n",
        "trestbps_calibrator = tfl.layers.PWLCalibration(\n",
        "    # Alternatively, you might want to use quantiles as keypoints instead of\n",
        "    # uniform keypoints\n",
        "    input_keypoints=np.quantile(training_data_df['trestbps'],\n",
        "                                np.linspace(0.0, 1.0, num=5)),\n",
        "    dtype=tf.float32,\n",
        "    # Together with quantile keypoints you might want to initialize piecewise\n",
        "    # linear function to have 'equal_slopes' in order for output of layer\n",
        "    # after initialization to preserve original distribution.\n",
        "    kernel_initializer='equal_slopes',\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[3] - 1.0,\n",
        "    # You might consider clamping extreme inputs of the calibrator to output\n",
        "    # bounds.\n",
        "    clamp_min=True,\n",
        "    clamp_max=True,\n",
        "    monotonicity='increasing',\n",
        "    name='trestbps_calib',\n",
        ")(\n",
        "    trestbps_input)\n",
        "lattice_inputs.append(trestbps_calibrator)\n",
        "\n",
        "# ############### chol ###############\n",
        "chol_input = tf.keras.layers.Input(shape=[1], name='chol')\n",
        "model_inputs.append(chol_input)\n",
        "chol_calibrator = tfl.layers.PWLCalibration(\n",
        "    # Explicit input keypoint initialization.\n",
        "    input_keypoints=[126.0, 210.0, 247.0, 286.0, 564.0],\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[4] - 1.0,\n",
        "    # Monotonicity of calibrator can be decreasing. Note that corresponding\n",
        "    # lattice dimension must have INCREASING monotonicity regardless of\n",
        "    # monotonicity direction of calibrator.\n",
        "    monotonicity='decreasing',\n",
        "    # Convexity together with decreasing monotonicity result in diminishing\n",
        "    # return constraint.\n",
        "    convexity='convex',\n",
        "    # You can specify list of regularizers. You are not limited to TFL\n",
        "    # regularizrs. Feel free to use any :)\n",
        "    kernel_regularizer=[('laplacian', 0.0, 1e-4),\n",
        "                        tf.keras.regularizers.l1_l2(l1=0.001)],\n",
        "    name='chol_calib',\n",
        ")(\n",
        "    chol_input)\n",
        "lattice_inputs.append(chol_calibrator)\n",
        "\n",
        "# ############### fbs ###############\n",
        "fbs_input = tf.keras.layers.Input(shape=[1], name='fbs')\n",
        "model_inputs.append(fbs_input)\n",
        "fbs_calibrator = tfl.layers.CategoricalCalibration(\n",
        "    num_buckets=2,\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[5] - 1.0,\n",
        "    # For categorical calibration layer monotonicity is specified for pairs\n",
        "    # of indices of categories. Output for first category in pair will be\n",
        "    # smaller than output for second category.\n",
        "    #\n",
        "    # Don't forget to set monotonicity of corresponding dimension of Lattice\n",
        "    # layer to '1'.\n",
        "    monotonicities=[(0, 1)],\n",
        "    # This initializer is identical to default one ('uniform'), but has fixed\n",
        "    # seed in order to simplify experimentation.\n",
        "    kernel_initializer=tf.keras.initializers.RandomUniform(\n",
        "        minval=0.0, maxval=lattice_sizes[5] - 1.0, seed=1),\n",
        "    name='fbs_calib',\n",
        ")(\n",
        "    fbs_input)\n",
        "lattice_inputs.append(fbs_calibrator)\n",
        "\n",
        "# ############### restecg ###############\n",
        "restecg_input = tf.keras.layers.Input(shape=[1], name='restecg')\n",
        "model_inputs.append(restecg_input)\n",
        "restecg_calibrator = tfl.layers.CategoricalCalibration(\n",
        "    num_buckets=3,\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[6] - 1.0,\n",
        "    # Categorical monotonicity can be partial order.\n",
        "    monotonicities=[(0, 1), (0, 2)],\n",
        "    # Categorical calibration layer supports standard Keras regularizers.\n",
        "    kernel_regularizer=tf.keras.regularizers.l1_l2(l1=0.001),\n",
        "    kernel_initializer='constant',\n",
        "    name='restecg_calib',\n",
        ")(\n",
        "    restecg_input)\n",
        "lattice_inputs.append(restecg_calibrator)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Fr0k8La_YgQG"
      },
      "source": [
        "次に、キャリブレータの出力を非線形に融合するラティスレイヤーを作成します。\n",
        "\n",
        "必要とされる次元で増加するラティスの単調性を指定する必要があることに注意してください。キャリブレーションで単調性の方向を構成することにより、単調性のエンドツーエンドの方向が正しくなります。これには、`tfl.layers.CategoricalCalibration`レイヤーの部分的な単調性が含まれます。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "X15RE0NybNbU"
      },
      "outputs": [],
      "source": [
        "lattice = tfl.layers.Lattice(\n",
        "    lattice_sizes=lattice_sizes,\n",
        "    monotonicities=[\n",
        "        'increasing', 'none', 'increasing', 'increasing', 'increasing',\n",
        "        'increasing', 'increasing'\n",
        "    ],\n",
        "    output_min=0.0,\n",
        "    output_max=1.0,\n",
        "    name='lattice',\n",
        ")(\n",
        "    lattice_inputs)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "31VzsnMCA9dh"
      },
      "source": [
        "モデルに柔軟性を追加するために、出力キャリブレーションレイヤーを追加します。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "efCP3Yx2A9n7"
      },
      "outputs": [],
      "source": [
        "model_output = tfl.layers.PWLCalibration(\n",
        "    input_keypoints=np.linspace(0.0, 1.0, 5),\n",
        "    name='output_calib',\n",
        ")(\n",
        "    lattice)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1SURnNl8bNgw"
      },
      "source": [
        "入力と出力を使用してモデルを作成できるようになりました。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7gY-VXuYbZLa"
      },
      "outputs": [],
      "source": [
        "model = tf.keras.models.Model(\n",
        "    inputs=model_inputs,\n",
        "    outputs=model_output)\n",
        "tf.keras.utils.plot_model(model, rankdir='LR')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tvFJTs94bZXK"
      },
      "source": [
        "トレーニングは、他の Keras モデルと同じように機能します。このセットアップでは、入力された特徴が個別のテンソルとして渡されることに注意してください。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vMQTGbFAYgYS"
      },
      "outputs": [],
      "source": [
        "feature_names = ['age', 'sex', 'cp', 'trestbps', 'chol', 'fbs', 'restecg']\n",
        "features = np.split(\n",
        "    training_data_df[feature_names].values.astype(np.float32),\n",
        "    indices_or_sections=len(feature_names),\n",
        "    axis=1)\n",
        "target = training_data_df[['target']].values.astype(np.float32)\n",
        "\n",
        "model.compile(\n",
        "    loss=tf.keras.losses.mean_squared_error,\n",
        "    optimizer=tf.keras.optimizers.Adagrad(LEARNING_RATE))\n",
        "model.fit(\n",
        "    features,\n",
        "    target,\n",
        "    batch_size=BATCH_SIZE,\n",
        "    epochs=NUM_EPOCHS,\n",
        "    validation_split=0.2,\n",
        "    shuffle=False,\n",
        "    verbose=0)\n",
        "\n",
        "model.evaluate(features, target)"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "keras_layers.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
